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Abstract 

This paper describes a multidimensional program of student 
assessment that has been in place for over seven years at Ohio 
University. Various forms of value-added testing, measures of 
social and academic integration, and alumni outcomes measures are 
used. The program is unique in that general education 
requirements have changed during the seven years of the program, 
and valuable pre- and post-tests have been conducted. 
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A Multidimensional Approach to Student Outcomes Assessment 
The assessment of institutional impact on students is viewed 
as critical to the evaluation of academic programs. Many 
national reports have discussed the subject in great devail over 
the past two years. Numerous attempts have been made to define 
student outcomes (Ewell, 1984; Chickering, 1979; Lenning, 1977; 
Pace, 1984;), and a great number of research projects have been 
conducted (e.g., Banta, 1984; McClain and Krueger, 1984) or 
recommended (e.g., Study Group on the Conditions of Excellence in 
American Higher Education, 1984). 

This paper describes an institutional commitment of Ohio 
University to assess its impact on its students. This commitment 
was developed as a result of the 1980 State of the University 
address delivered by President Charles J. Ping to Ohio University 
faculty, students, and staff. The speech, entitled "Quality 
Dependent on the Making of Judgments," called for the University 
to make a commitment to growth in quality. In making this 
commitment the University, as a part of the ten-year plan, had a 
responsibility to define and examine this growth in quality. In 
his speech the president stated that to be of value this 
assessment of quality must have consequences, and offer direction 
for decision making and action. 

As part of his address. President Ping appointed an 
institutional impact task force of faculty and staff to meet with 
the director of institutional research to develop a coherent, 
systematic, and ongoing program of assessment of institutional 
impact. The primary goal of this assessment was that in five to 
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ten years the University could systematically describe what was 
happening in its life as an educational institution. 

The institutional impact task force spent about one year 
developing a program for assessing the impact of the college 
experience on students at Ohio University. This program 
considered such goals in the "Ten-year Educational Plan" as 
"providing our students with the knowledge and skills which are 
the essence of a solid liberal education and encouraging the 
development of an environment on the residential campus that 
reflects a vital commitment to learning and provides a community 
life for students." To accomodate these basic goals, our 
assessment was aimed at the broad impact of the collegiate 
experience and not at specific knowledge and skills gained from a 
major field of study. The outcomes program provided a 
broad-based yet flexible set of measuring devices. They were 
broad in the sense of allowing for a sufficiently diverse set of 
measures that enable a breadth of information to be gathered and 
flexible with respect to its acceptance of new instruments as 
they become available and rejection of instruments as they become 
dated. At the end of the one year period of study the committee 
reported back to the president a recommended program for 
conducting institutional assessment. This recommended program 
then became the theme for the 1981 State of the University 
address thar was reported to faculty, students, and staff. 

Program Description 

To provide a variety of measures of assessment at various 
points in time a multidimensional program was proposed and 
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accepted beginning fall quarter 1981. This plan made use of both 
nationally developed assessment devices as well as locally 
developed instruments. Figure 1 presents a diagram of the 
complete assessment system at Ohio University. 

To measure the impact of general education the American 
College Testing (ACT) Program College Outcomes Measures Project 
(COMP) instrument was used. The program originally used both the 
COMP/ACT Composite exam and the COMP/ ACT Objective exam. These 
two instruments are nationally normed and designed to assess 
student abilities in the following areas: communicating, solving 
problems, clarifying values, functioning within social 
institutions, using science and technology, an using the arts. 
The COMP/ACT Composite exam provides scores on writing, 
computing, and reading abilities. The Objective exam serves as a 
proxy measure of the composite exam and can be administered to a 
larger number of students in a shorter period. During the first 
year of the study both the Composite and Objective exam were 
used, but in all following years only the Objective exam was 
used. This testing was carried out on a longitudinal basis, and 
data have been collected on seven classes. 

Another instrument designed to monitor change over the 
college career of the student is the Student Involvement 
Questionnaire. This instrument was originally designed by the 
Center for Helping Organizations Improve Choice in Education 
(Project CHOICE) , in which Ohio University participated in 1978. 
When the institutional assessment program began in 1981 this 
instrument was modified and has been given to each freshman and 
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senior class since 1983. The modified questionnaire collects 
information on three areas of involvement: academic involvement, 
social involvement and activities, and students' goals and 
commitment to the institution • It collects information about 
students' patticipation in activities, organizations, 
extracurricular activities, contact with faculty, staff, and 
other students, and it assesses attitudes about satisfaction with 
Ohio University. This questionnaire is also used to idertify 
potential "drop-outs" while they are still enrolled. 

Anothor important component of the assessment system is the 
Student Tracking System. This system utilizes a cohort survi^^al 
method of tracking each freshman until they either graduate or 
drop out of the University. This system's purpose is to identify 
potential areas of high attrition and to longitudinally monitor 
progress with the University's retention programs. 

The Student Treatment Study is a part of the assessment 
process that was started before 1981, discontinued, and now has 
been reconstituted. This study collects information on the 
attitudes and perceptions of new freshmen after one quarter of 
enrollment, concerning the campus and its environment. It 
includes sectionji on treatment received from other people, 
quality of information students receive, and perceptions of 
processes that freshmen experience while at Ohio University. 
This instrument is administered every two years, and change is 
plotted from one freshman class to another. 

The final area of outcomes information collected by Ohio 
University includes placement and alumni studies. The University 
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had varying degrees of success in collecting this information 
before the institutional impact project, as different colleges 
and departments used various instruments to collect data that 
could be used for within departments but did not allow for 
interdepartmental comparison. As part of the design of the 
institutional impact project two separate instruments were 
developed. The first, a placement survey, was designed to 
collect information on graduates within one year of graduation. 
It collects information on employment status, graduate or 
professional school acceptance, and some limited information on 
employment satisfaction. The instrument was developed locally, 
and unsuccessful attempts have been made to have peer 
institutions use a similar instrument and share placement 
results. 

A more complete graduate survey, the Alumni Survey, was 
developed to survey graduates after they had been away from the 
University for at least five years. The questionnaire was 
developed from scales of an instrument used by the Associated 
Colleges of the Midwest, a Project CHOICE instrument, and locally 
developed questions from Ohio University's eight academic 
colleges. 

The questionnaire was divided into three major sections • The 
first section asked questions about traditional alumni outcomes, 
such as type of job held, employment status, salary, employment 
satisfaction, nature of employment responsibilities, relevance of 
educational experience to employment, and problems in seeking 
employment. There was also a group of questions dealing with 
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various coippetencies needed and developed. The second section 
asked questions about graduates' programs of study and their 
current satisfaction with their undergraduate programs. 
Questions asked about additional degrees received or hoped for^ 
ratings of the education received at Ohio University, and changes 
that graduates would make in their education if they were given 
the chance to do it over again. The third section of the 
questionnaire contained college-specific questions, which 
included questions specific to each college's programs. In th^ 
last two years of the alumni study, each of the eight acadenic 
colleges participated in questionnaire preparation. 

These six components — COMP/ACT, Student Involvement, Student 
Tracking, Student Treatment, Placement Survey, and Alumni 
Survey — currently comprise the Ohio University student outcomes 
assessment system. The system is evaluated on a regular basis, 
and instruments are added and modified in areas that require 
further attention. One of the system's original purposes, to be 
flexible enough to meet future needs and recognize areas that are 
no longer appropriate, is still important in Ohio University's 
evaluation of assessment devices. 

Data Collection Procedures 

Value-Added Testing 

Ohio University has used the two and one-half hour COMP 
Objective test since 1981, testing random samples of freshmen and 
seniors until 1984, when longitudinal testing of seniors was able 
to be done. About 1,200 freshmen and 600 seniors have been 
tested. Three years of longitudinal data have been collected 
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since 1981. 

New freshmen are sampled in the beginning of each fall 
quarter. Students are sent a personal letter from the president 
of the University that describes the program and the dates and 
times that the test is offered. A letter from the institutional 
research office accompanies the president's letter. It describes 
the program in more detail, and it offers an incentive for 
student participation. Freshmen are offered a coupon for a small 
pizza (about a $4.00 value) for their participation. A variety 
of times are offered for students to take the test, all within 
one week. "Make-up" sessions are offered about two weeks later 
for tnose studp.nts who could not attend the first session. 
Between 200 and 250 freshmen actually participate each year. 

For the last three years Ohio University has had the 
opportunity to retest seniors who took the COMP as freshmen. 
Students who are still enrolled and who took the test as a 
freshman four years before, five years before (and missed the 
previous year's senior test), and students who took the test as a 
freshman three years before (and are not likely to be enrolled 
the next year because of early graduation) are invited. Seniors 
receive a personal letter from the president and a letter from 
the institutional research office describing the program and the 
incentive that is offered. Seniors are offered a $10.00 gift 
certificate from a local restaurant for their participation. A 
variety of times are offered for seniors to participate. The 
letter requests that students call the institutional research 
office to schedule which session they will attend. Seniors who 
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have not contacted the institutional research office are called 
personally and asked to participate. The senior response rate is 
about 70 percent each year. 
Social and Academic Integration 

The Student Involvement Study has been conducted 
cooperatively by the Institutional Research Office and the Dean 
of Students Office. Each year members of these two offices meet 
(in the form of the Student Affairs Research Advisory Committee) 
to discuss modifications in the freshman instrument and the data 
collection procedure. Suggestions for changes are considered, 
but the majority of the instrument is left unchanged so that 
information can be collected on trends from year to year. 

The information that is collected has been used in three 
ways. First, a year-to-year comparison of freshmen has been 
done. Second, freshman to senior comparisons have been made. 
Third, the information is used as part of a retention program. 
Individual freshman responses are used to identify potential 
leavers each year so that residence life staff can intervene and 
help solve individuals' attrition-related problems. 

In 1979, 1983, 1984, 1985, and 1986 freshmen at Ohio 
University have p2i.ticipated in the freshman Involvement Study. 
In 1979 a random sample of freshmen were sent a questionnaire, 
and in subsequent years the entire classes of freshmen living in 
the residence halls (about 90 percent of the freshman class) have 
been surveyed. Questionnaires are distributed through resident 
assistants during the first week of spring quarter. However, 
much preparation for that distribution takes place. After the 
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final form of the questionnaire is decided, meetings take place 
with Institutional Research, Dean of Students, and Residence Life 
staff. These ""-^■^ings are designed to involve the residence life 
staff in the d^^a collection and student intervention process. 
Additional meetings, raailings, and reminder notices occur from 
the dean of students office to the residence hall directors to 
further elicit their support. 

Freshman response rates have varied over the years because of 
changes in the distribution process. The freshman response rates 
are as follows: 1979, 53 percent; 1983, 43 percent; 1984, 90 
percent; 1985, 75 percent; 1986, 49 percent; 1987, 68 percent. 
Experience has shown that the greater the involvement of the 
residence life staff (resident assistants and residence hall 
directors), the greater the response rate has been. 

Two follow-up studies of seniors who were surveyed as 
freshmen have been done, and the third year of senior data is 
currently being collected. Freshmen in the 1978-1975, 1982-1983, 
and 1983-1984 classes have been surveyed as seniors. Seniors who 
participated as freshmen are asked to participate again, and they 
are given a questionnaire that is identical to the one they 
completed as freshmen. In the spring quarter of their senior 
year they receive a personal letter from the president of the 
University and a note from the institutional research office 
that describes the program and the incentive that is offered. 
The current incentive that is offered is a coupon for free ice 
cream cone from the local Baskin-Robbins store. Other incentives 
that have been offered in the past have been free admission to a 
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local theater, a free hamburger, and free admission to an 
on-campus pub where local entertainment performs. Tie response 
rates of the seniors were 91 percent in the first year and 77 
percent in the second year. 

The Freshman Treatment Study was first dona for the 1978 
freshman class and was replicated for the 1984 freshman class. 
The study was adapted from work done at the University of 
Missouri at Columbia, and it was used at Ohio University to 
identify problem areas in student services and academic areas. 
The questionnaire was designed to measure how students felt they 
were treated by the staff of various offices, the quality of 
information they received from those offices, and their 
perceptions of processes they were exposed to at Ohio University. 
It used a semantic differential scale to measure students' 
ratings on such things as accessibility, courtesy, and 
efficiency; it solicited comments and suggestions about the 
University offices and processes that freshman deal with; and it 
asked whether the student intended tc " aave the University before 
finishing the freshman year and what factors influenced their 
decision. Random samples of freshmen were collected each year; 
the response rate was 39 percent in each year. 

The Institutional Research Office is currently in the process 
of completing the Treatment Study for each of its six branch 
campuses . 
Alumni Outcomes 

Two surveys were designed to address alumni outcomes. A 
placement survey is sent to every graduate (undergraduate, 
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graduate, and professional degrees) within one year of 
graduation. A more extensive alumni survey is sent to 
undergraduate degree recipients within five and ten years of 
graduation. 

The Placement Study has been conducted cooperatively by the 
Institutional Research Office and the Career Planning and 
Placement Office. The Placement Study has been completed for 
graduates of the classes of 1980, 1984, 1985. Data from the 1986 
class is currently being collected. It collected information 
through a questionnaire that was mailed to graduates within one 
year of graduation. In 1980 the questionnaire was mailed to a 
randcm sample of 1,000 graduates from fall 1979 through summer 
1980. Forty-one percent of those sampled responded. In 1984 and 
1985 the questionnaire was mailed with diplomas to all graduates. 
About 40 percent of the graduates responded to the Placement 
Study. Questionnaires are no longer sent with diplomas because 
that was too soon after graduation to get accurate employment 
information. Questionnaires are now sent about one year after 
graduation, and the distribution of the questionnaires is closely 
monitored by institutional research staff. 

The Alumni Study has been conducted on graduates five to ten 
years after graduation. Since 1982 six classes have been 
surveyed. In the first four years the questionnaire was mailed 
to random samples of Ohio University alumni who had received the 
baccalaureate five and ten years earlier, respectively. In 1986 
the entire classes of 1978 and 1979 were surveyed. In 1982 the 
questionnaire was mailed to Ohio University alumni holding the 
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baccalaureate from the classes of 1971 and 1976. Random samples 
from the two classes were drawn, and questionnaires and personal 
letters from the president of the University were sent. The 
response rate was about 44 percent. In 1983 a random sample of 
graduates from the classes of 1&72 and 1977 were surveyed. The 
response rate was about 54 percent. 

In 1986 plans were male to survey 1978 and 1979 alumni. 
Several requests for alumni outcomes information had been 
received from academic departments, and it was determined that 
larger pools cf data were needed in order to satisfy these 
requests. In addition, each academic college was contacted and 
given the opportunity to include their own questions in a section 
on the alumni questionnaire. The entire classes of bachelor's 
degree recipients were surveyed. Results of the 1978 and 1978 
university-wide and college-specific surveys are currently being 
analyzed. 

Results 

Brief results are presented here to provide examples of how 
information is interpreted. 
Value-Added Testing 

The COMP Objective test has not been used to directly assess 
general education outcomes at Ohio University, but it has 
provided some useful evaluative material during a change in the 
general education program. The general education program 
underwent a review that was implemented in 1979, and the COMP was 
introduced at Ohio University in 1981. The Tier I (English 
composition and quantitative skills) component was required of 
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freshmen entering in 1979. The Tier II (breadth of knowledge or 
"distribution") component was required of freshmen entering in 
1981. A report of the "Tier II Review Committee" concludad that 
students had taken more "Third World Cultures" and "Applied 
Science and Technology" courses since the inception of Tier II. 
The Tier III (senior synthesis) was required of freshmen entering 
in 1982; these students were seniors in 1986, which was the first 
year the COMP was administered to students who had taken the Tier 
III requirement. Although it is unclear which seniors in 1986 
took the Tier III requirement before taking the COMP, it is 
likely that most of them had taken it before taking the COMP. 

The Office of Institutional Research has collected data on 
seniors who have taken the COMP since 1981. Three classes of 
seniors who took general education classes prior to the Tier 
system took the COMP test, and two classes of seniors who have 
taken most of the general education Tier requirements have taken 
the COMP test. Except for a slight decline in senior percentile 
scores from 1981-1982 to 1982-1983, the senior scores have 
improved steadily. Improvement is especially noted in the two 
senior classes who had taken the Tier requirements. In 1981-1982 
the total COMP score was at the forty-ninth percentile (based on 
a national sample of seniors who took the COMP) , in 1984-1985 it 
was at the fifty-eighth percentile, and in 1985-1986 the seniors' 
total COMP score was at the sixty- fourth percentile. The larger 
increase in 1986 seniors' scores may be attributable to the fact 
that some of these students had taken the Tier III requirement. 
The Tier III requirement is designed to teach students to apply a 
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variety of information to solving problems; this element is 
similar to the type of aptitude skills that the COM? measures. 

All of these seniors pursued their education at Ohio 
University while it was essentially an open-admission 
institution. The admission policy has since changed to be more 
selective, and use of the CCMP for senior and freshman classes 
may provide useful information about levels of students' 
abilities as they enter and leave Ohio University. 

The COMP has been administered to freshmen since 1981. The 
scores of freshmen who persisted to the senior year scored higher 
than fres^men who dropped out before their senior year. For 
example, 1982-1983 freshmen who persisted to the senior year in 
1985-1986 scored at the thirty-seventh (senior) percentile, while 
all freshmen tested in 1982-1983 scored at the twenty-ninth 
percentile. 

In 1984-1985 seniors took the COMP who took it in 1981-1982. 
These students achieved a 20 percentile point learning gain on 
the total score, which is comparable to the national average for 
learning gain. The greatest test subscore gains were in 
"clarifying values" and "using science and technology." The 
senior test percentiles ranged from 51 to 63, based on a national 
sample. In 1985-1986 seniors took the COMP who took it in 
1982-1983 as freshmen. These students achieved a 27 percentile 
point learning gain on the total score, which is above the 
national average for learning gain. The greatest test subscore 
gains were in "solving problems" and "us'ing the arts." The 
senior test percentiles for this group ranged from 55 to 76, 
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based on a national sample. 
Social and Academic Integration 

The Student Involvement questionnaire was given to freshmen 
in 1979, 1983, 1984, and 1985, and 1986. Freshmen in 1979 who 
vere seniors in 1983 were given the questionnaire again, and 
freshman and senior data were compared. The number of formal 
contacts (such as advising) and informal contacts (such as dinner 
with a faculty member) freshman had with faculty increased from 
1983 to 1985, and the number of contacts seniors had with faculty 
wafc» greater than the number of contacts they had as freshmen. 
F..eshmen reported greater satisfaction with faculty, the 
pre-college orientation, and Ohio University in general in 1985 
than they did in 1983 and seniors indicated greater satisfaction 
with Ohio University in general than they did as freshmen. The 
amount of time freshmen spent studying increased from 1983 to 
1985, and the amount of time seniors spent studying and the 
number of books read outside of class were greater than they were 
as freshmen. The number of weekends students spent per month on 
campus increased from 1983 to 1985. Involvement in 
extracurricular activities increased from 1983 to 1985. Seniors 
reported greater participation in student organizations than they 
did as freshmen, and they indicated that more of their best 
friends attended Ohio University than they did as freshmen. 

The Office of Institutional Research has regularly monitored 
the freshman attrition rate through its tracking system. The 
retention rate improved steadily since 1977. In 1977, 33.1 
percent of the freshmen did not return to Ohio University as 
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sophomores, and in 1985, 25.0 percent of them did not return as 
sophomores, which is lower than the national attrition rate. The 
attrition rate of students in the top quintile of their high 
school class declined from 19 percent in 1978 to 15 percent in 
1984, and the attrition rate of students with a college grade 
point average of 2.5 and above has declined. These figures, along 
with changes in predictor statistics in the Involvement Study, 
suggest that the identification and intervention strategy is 
having positive effects. 

In 1978 and 1984 freshmen were surveyed as to the equality of 
treatment they received from faculty and various campus offices. 
Most areas improved between 1978 and 1984, but the greatest 
improvements were in academic areas: advising, treatment by 
faculty, information about what to expect in the classroom, and 
information about the process of getting help with academic 
problems. Significant improvements in treatment of freshmen were 
reported in offices such as the Cashier's Office, the Housing 
Office, and by the Pre-College staff. 
Alumni Outcomes 

The Placement Study revealed that about 80 percent of the 
graduates were employed full-time at the time of the surveys, and 
most of them were employed in Ohio. Most of the graduates 
reported being satisfied with the academic preparation they 
received at Ohio University for their first job and for 
additional academic work (95 and 98 percent, respectively). 
Ninety-seven percent of the graduates reported being satisfied 
with the preparation they received at Ohio University for 
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establishing career goals. 

Alumni Study questionnaires were sent to five- and ten-year 
graduates. The study collected information on employment^ 
employment satisfaction, and competencies needed and developed at 
Ohio University. Graduates reported salaries that averaged 
higher than national and state salary averages. No more than 2.5 
percent of the graduates expressed any dissatisfaction with their 
major programs, general education, elective courses, or 
extracurricular activities. Graduates reported several 
competencies, from a list, that were most needed and best 
developed at Ohio University. These competencies were the 
ability to acquire new skills and understanding, the ability to 
think analytically, the ability to formulate creative and 
original ideas and solutions, sensitivity to feelings and 
perceptions of others, and the ability to communicate orally. 

Implications and Uses 

The use of outcomes information receives a high priority as 
highlighted by the University's president in his 1980 State of 
the University address, in which he stated that "the key issue is 
whether we can use the [assessment] process of defining and 
examining to enhance quality." 

The president has established a policy that at each fall 
quarter meeting of the Board of Trustees an in-depth report is 
made of one of the dimensions of the assessment program, and a 
general report is given on the status of the overall project. 
This report receives special interest from the trustees, and many 
provoking and intuitive questions are asked by them. 
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The National Alumni Board also receives a major briefing on 
some aspect of the assessment system each year from the 
institutional research office. In addition, the alumni relations 
director forwards to all members of the National Alumni Board all 
reports he receives produced as a part of the assessment system. 

On campus all major planning groups are involved in receiving 
various types of information concerning the assessment system. 
The Dean's Council receives reports on general education, student 
involvement, retention, and placement and alumni. In addition, 
each college receives their own personalized reports which apply 
directly to their specific planning needs. 

The University Planning Council receives periodic briefings 
on the assessment system and other large groups, including the 
faculty senate, the curriculum review council, and the university 
advising council receive both oral and written reports on varying 
aspects of the assessment system. 

Individual campus offices are targeted for receiving most of 
the information produced from the assessment system. These 
offices include: Career Planning and Placement, Alumni 
Relations, Admissions, Student Affairs, Residence Life, and the 
Counseling and Guidance Center. 

Specific assessment information has been used to develop or 
modify a number of programs across campus. Attrition assessment 
information is used by colleges and departments to develop 
retention programs. For example, the Student Involvement Study 
has been used to develop a prediction system to identify 
individual potential drop-outs with a grade point average of at 
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least 2.0 and intervene while they are still enrolled. Both 
academic and residence life staff are involved in this effort to 
contact and retain these students. Information from the 
Treatment Study was used to establish a new advising system for 
undecided students in the University College, and the Tracking 
system gives each college an annual update of how they are 
progressing with their individual retention programs. 
Information from all these assessment systems has contributed 
greatly to the reduction of 33 percent of freshmen leaving during 
their freshman year in 1978 to only 25 percent of the freshmen 
leaving in 1986. 

The Placement Survey is being used to evaluate the success of 
the Career Planning and Placement Office and to provide useful 
information to incoming students. The Admissions office and 
individual colleges use this information for prospective 
freshmen. The Alumni Study is being used to evaluate the success 
of academic programs on a long-term basis and to identify areas 
of competencies that are important to Ohio University graduates. 
In addition, college-specific alumni information is assisting 
individual colleges with program review and accreditation. Both 
placement and alumni data have been useful in recent Program 
Excellence awards given by the Ohio Board of Regents. Within the 
last three years '^hio University has won four of these awards for 
about $650,000 for four "excellent" academic programs. 

Ohio University's revised general education program is being 
evaluated, and one of the measures of its success will be our 
longitudinal testing of freshmen and seniors who experienced the 
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old and new general education requireir.ants. 

In 1983 much of our assessment information was included in 
the comprehensive review of the North Central Association 
accreditation review process. While current criticisms of 
institutional reviews by accreditation associations have pointed 
to a weakness in assessment data, at Ohio University the North 
Central Association review team suggested that instead of not 
having enough assessment data, we may have too much. This is a 
perplexing problem that institutions must manage as the mountains 
of assessment data begin to accumulate at our institutions of 
higher education in the near future. 
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